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SUMMARY 

With  the  increasing  use  of  uninhabited  military  vehicles  in  air,  land,  naval  (surface  and  underwater)  roles, 
we  need  to  know  more  about  factors  affecting  operator  "engagement"  -  cognitive/conative/behavioural 
task  involvement  -  with  systems  exploiting  virtual  media  technology’,  in  particular  for  reach-back,  remote 
supervision  of  operations  involving  use  of  lethal  force.  UK  experience  in  operating  Predator  has  led  to 
concerns  about  the  operator  needing  emotional  connectivity  to  "feel  the  granularity  of  the  battle-space", 
about  the  "morality  of  altitude",  and  the  potential  for  the  "playground  bully"  to  become  the  mode  of 
control.  Research  has  demonstrated  the  difficulty  of  providing  sustained  levels  of  cognitive  engagement 
for  operators  at  remote  control  stations  providing  supervisory  targeting  veto.  It  may  be  possible  to 
mitigate  these  risks  and  to  augment  human  involvement  and  engagement  strategies  through  operator 
selection,  training  and  system  design.  Consideration  is  needed  of  the  relevance  of  mission  and  decision 
enabling  technologies  for  augmenting  engagement.  These  enabling  technologies  include  advanced  human- 
computer  interfaces,  virtual  media,  multi-modal  “immersive”  synthetic  environments,  task  and  user 
monitoring  and  modelling,  collaborative  technologies  and  communication  techniques  such  as  semantic 
information/knowledge  web  approaches  to  decision  effectiveness. 


1.  BACKGROUND 

Virtual  media  studied  from  a  socio-technical  systems  or  human  factors  (HF)  perspective,  considers  virtual 
technology  as  tools  for  human  use,  providing  enabling  and  augmentation  of  human  capability.  In  the 
military  environment,  virtual  media  technology  is  exploited  extensively  for  training  purposes  and 
increasingly  for  prosecuting  military  operations  under  Network- Centric  Warfare  (NCW)  concepts.  A 
potentially  useful  framework,  arising  from  tele-operations  research  (Leo  van  Breda,  TNO,  personal 
communication),  is  to  consider  virtual  media  by  distinguishing  between  virtual  presence  (e.g.  user 
emersion,  synthetic  visualisation  of  agents,  robotic  behaviour),  virtual  cognition  (e.g.  automatic 
detection/recognition  systems,  machine  or  artificial  intelligence  (AI),  decision  systems,  agent  oriented 
software),  and  virtual  communication  (e.g.  digitaFelectronic  vision  and  telecoms  systems,  synthetic 
speech,  3-D  vision/sound,  tactile  interfaces,  head  mounted  displays).  A  rapidly  emerging  NCW 
technology  that  exploits  all  of  these  dimensions  of  virtual  media,  both  in  military  training  and  operations, 
is  Uninhabited  Air  Vehicle  (UAV)  systems.  In  one  recent  research  implementation,  the  US  AFRL 
“Vigilant  Spirit”  Research  UAV  Control  Station  (UCS)  supports  supervision  of  automatic  air-to-air 
refuelling  of  multiple  UAVs  by  presenting  the  operator  with  variable  synthetic  views  of  the  positioning  of 
the  UAVs  with  respect  to  synthetic  views  of  refuelling  tanker  aircraft  (Mark  Draper  and  Greg  Feitshans, 
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US  AFRL/HE,  personal  communication).  UAVs  are  a  class  of  Uninhabited  Military  Vehicles  (UMVs).  HF 
of  UMVs  have  been  studied  under  RTO  F1FM-078/RTG-017  Technical  Team.  This  paper  seeks  to 
consider  F1F  of  virtual  media  in  the  context  of  the  UAV/UMVs,  as  investigated  by  RTO  HFM-078/RTG- 
017,  and  in  particular  the  military  relevance  of  UAV/UMVs.  The  Final  Report  of  F1FM-078/RTG-017, 
currently  in  press,  will  provide  a  broader  and  more  detailed  analysis  of  these  UMV  F1F  issues  [1]. 

UAVs  are  enablers  of  military  capability  with  clear  endorsement  at  the  highest  level.  Many  NATO 
Nations  have  active  programmes  to  develop  and  integrate  UAV  systems  into  the  front  line  military  force 
mix.  UAVs  are  most  commonly  characterised  as  dealing  well  with  3-D  tasks  -  dull,  dirty  and  dangerous. 
They  are  used  extensively  in  intelligence,  surveillance  and  reconnaissance  (ISR)  roles,  affording 
persistence  in  the  provision  of  critical  information,  without  risking  lives.  Increasingly,  they  are  being 
considered  for  combat  and  support  roles.  Modem  warfare  needs  military  capability  to  respond  to  the  threat 
of  conventional  hostile  force  and  to  the  challenges  of  asymmetric  conflicts,  where  political  and  military 
success  relies  on  effects-based  targeting  and  operations.  In  the  age  of  NCW,  or  Network  Enabled 
Capability  (NEC),  ISR  information  supplied  by  UAV  systems  can  be  a  key  combat  multiplier  in  the  hands 
of  a  commander.  Automation  technology  and  computer-based  information  processing  are  increasingly 
important  for  balancing  affordability,  capability  and  achievability  with  increasing  pressures  on  scarce, 
skilled  human  resources.  Important  questions  remain  about  what  realistic  effects  can  be  expected  to  be 
achieved  by  UAVs  in  the  uncertain,  ambiguous  and  non-linear  battle-space  of  the  future,  including  how 
international  law  will  inteipret  robotic  warfare  in  the  future.  However,  the  main  consideration  of  interest  is 
not  so  much  the  military  relevance  of  UAVs,  since  this  seems  mostly  self  evident.  Rather,  the  key  issue  is 
to  establish  why  HF  are  important  military  relevant  issues  with  “unmanned”  technologies.  Since  UAV 
technologies  are  expected  to  actually  reduce  human  involvement  in  some  tasks,  it  is  not  self  evident  why 
HF  issues  should  warrant  raised  attention. 

2.  UAV  HUMAN  FACTORS 

UAVs  are  valued  variously  as  force-multipliers,  as  augmenters  of  the  force,  and  as  adding  a  new 
component  to  the  military  force  mix.  But  ultimately,  UAVs  are  tools  for  human  use.  Human  effectiveness 
is  the  key  to  all  military  capability.  UAVs  are  enablers  of  human  effectiveness  and  military  capability. 
Human  involvement  in  UAV  systems  is  of  paramount  importance.  HF  issues  need  to  be  in  the  sharpest 
focus  to  mitigate  the  unacceptable  risks  of  de-humanisation  of  decision-making  in  warfare. 

Human  factors  issues  are  crucial  to  the  successful  operation  of  UAV  systems.  Recent  USAFRL  flight 
safety  analysis  data  show  HF  to  be  a  causal  factor  in  70%  of  UAV  accidents.  HFM-078/RTG-017 
identified  that  force  multiplication  can  be  achieved  by  addressing  the  following  human  factors  issues  and 
challenges  for  UMV  systems  including  UAVs: 

•  Collaborative  Work  -  Optimal  Task  Distribution 

•  Virtual  team  performance 

•  Manned/Unmanned  collaboration 

•  Interoperability 

•  Flexible  level  of  automation 

•  Optimization  of  operator/vehicle  ratio 

•  Control  Stations  -  Intelligent  Operator  Support 

•  Operator  functional  state  assessment 

•  Intelligent  adaptive  interfaces 

•  Cognitive  cooperation 

•  Knowledge  management  systems 

From  a  high  level  military  perspective,  Air  Chief  Marshall  Sir  Brian  Burridge,  Commander  in  Chief, 
Strike,  of  the  Royal  Air  Force  (RAF)  believes  that  in  order  to  appreciate  the  capability  of  UAVs,  we  need 
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to  appreciate  their  limitations  and  benefits,  but  that  understanding  of  the  human  dimension  is  the  most 
important  of  all  -  knowing  how  to  use  them,  task  them  and  to  integrate  them  [2],  Use  of  UAVs  is 
generally  justified  on  grounds  of  capability,  affordability  and  safety.  UAVs  can  make  certain  tasks  safer 
by  reducing  human  involvement  and  risk  to  life,  allowing  the  possibility  of  human  resources  being  re¬ 
deployed  more  efficiency  and  effectively.  This  produces  complex  changes  in  the  balance  and  priority  of 
HF  issues  for  UAV  systems.  Paradoxically,  for  many  aspects  of  UAV  system  engineering  and  operation, 
the  proper  consideration  of  HF  has  even  greater  military  relevance.  Human  involvement  remains  essential 
throughout  the  UAV  system  life  cycle,  including  UAV  operations.  As  a  discipline,  HF  provides  the  tools 
for  understanding  and  ensuring  the  correct  human  involvement  in  the  UAV  system  life  cycle.  Obviously, 
UAV  habitability  is  not  a  concern.  However,  vehicle  maintenance  is  still  needed.  Vehicle  control  and 
safety  becomes  a  complex  issue,  especially  when  mixing  UAVs  with  manned  vehicles  and  “dismounted” 
forces.  Increasing  levels  of  UAV  autonomy  are  expected  to  reduce  the  need  for  human  intervention  in 
operations.  However,  UAVs  are  not  a  substitute  for  human  involvement  in  the  battle-space.  Crucially, 
human  control  of  UAVs  is  axiomatic  for  military  relevance.  Consideration  of  the  technological  viability  of 
autonomous  systems,  and  the  legal  constraints,  suggests  that  a  “human-in-the-loop”  system,  with  virtual 
media  providing  the  human-UAV  interface,  will  be  the  most  likely  mode  of  operation  to  provide  the 
required  supervision  and  discrimination. 


3.  SUPERVISORY  CONTROL 

In  warfare,  the  problems  and  outcomes  are  complex,  dynamic,  uncertain  and  risky,  and  the  application  of 
critical  judgement  and  decision  making  is  crucial  to  successful  conflict  resolution.  Context  sensitivity  is 
important  for  assessing  the  quality  of  military  decision  making.  Humans  encode  context  naturally  and 
handle  decision  making  adaptively  with  incomplete,  partial  and  uncertain  information.  This  provides 
decision  making  capability  not  easily  matched  by  AI  in  computers,  or  supported  by  virtual  media. 
However,  to  exercise  good  military  judgement,  humans  need  to  feel  the  texture  and  “granularity  of  the 
battle-space”.  Achieving  the  required  level  of  immersion  and  engagement  remotely  with  virtual  media 
presents  significant  technical  challenges  with  validation  and  measurement  issues,  which  must  ultimately 
be  judged  by  the  effectiveness  of  resultant  human  decision  making.  UAV  operators  removed  from  the 
immediate  context  of  use,  and  dependent  on  virtual  media  for  communication  and  presence,  risk  losing 
operator  context  sensitivity  and  system  adaptiveness.  Air  Chief  Marshall  Sir  Brian  Burridge  refers  to  this 
as  the  need  for  “emotional  connectivity  of  the  battle-space”  [2].  For  autonomous  UAV  operations, 
dependent  on  virtual  cognition,  the  detailed  level  of  operator  supervision  required  is  likely  to  be  dependent 
on  the  individual  mission  context  and  the  Rules  of  Engagement  (RoE).  This  can  be  difficult  or  impossible 
to  anticipate  fully  in  advance.  As  a  minimum,  the  operator  needs  virtual  media  to  be  enable  him/her  to 
discriminate  between  what  is  a  valid  military  target  and  what  is  not. 

Technological  limitations  and  legal  and  moral  constraints  suggest  that  some  form  of  human-in-the-loop 
control  always  will  be  required  [1,2].  Currently,  with  manned  vehicles,  the  human  operator  provides  the 
flexibility  to  adapt  to  constraints  on  functioning  arising  from  system  design,  creates  on-line  tolerance  of 
variability  and  uncertainty  in  the  external  environment,  and  offers  adaptation  to  changing  dynamic  mission 
context.  The  requirements  for  human-in-the-loop  control  of  UAV  operations,  either  remote  or  reach-back, 
can  be  considered  as  occurring  at  a  number  of  levels  depending  on  the  level  of  automation  e.g.  tasks, 
functions,  tactical  and  strategic  mission  goals.  Classes  of  control  can  be  characterised  as  either  manual, 
semi-automatic,  and  fully  autonomous  (cf  virtual  presence  with  virtual  cognition),  with  and  without 
human  supervisory  control.  Generally,  automation  best  serves  human  purposes  by  enabling  higher  levels 
of  human  control  i.e.  automate  routine  3-D  tasks  and  support  human  supervisory  control  at  tactical  and 
strategic  levels.  The  challenge  is  to  determine  the  precise  level  of  supervision  required,  and  to  identify  the 
detailed  user  requirements  and  HF  engineering  solutions,  for  efficient  and  effective  supervisory  control 
[3], 
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In  highly  autonomous  operations,  communications  permitting,  humans  can  retain  high  level  supervisory 
control  through  setting  and  monitoring  of  tasks  and  goals,  and  through  authorisation  of  safety  critical 
actions  and  use  of  lethal  force.  However,  experience  of  automation  supervision  elsewhere,  in  particular  in 
the  process  control  industry  and  with  flight  deck  automation  has  shown  that  reliable  and  robust  human 
supervisory  control  is  inherently  difficult  to  achieve.  Dependence  on  human  supervisory  control  is  risky 
for  safety  critical  events  and  tasks.  Limitations  on  cognition  (perception,  learning,  memory  and  reasoning) 
mean  that  it  is  inherently  difficult  for  humans  to  perform  supervisory  control  in  a  consistent  and  reliable 
manner,  particularly  during  sustained  operations  requiring  vigilance  and  unpredictably  intermittent  high 
levels  of  attentional  engagement.  Ultimately,  there  is  a  risk  that  the  over-use  of  automation  may  reduce 
human  authority,  responsibility  and  competency.  Crucially,  over-use  of  automation  risks  de-skilling  the 
user  in  the  important  cognitive  domain,  reducing  the  essential  human  capability  for  exercising  critical 
judgement  and  decision  making  in  the  appropriate  use  of  lethal  force. 

Supervisory  control  requires  robust  and  reliable  communications  with  the  battle-space.  The  realities  of 
military  communications  present  a  real  dilemma  for  the  supervisory  control  paradigm.  In  practice, 
communications  technology  limitations  (e.g.  line-of-sight  and  bandwidth  restrictions,  information  quality, 
latency)  and  communications  breakdown  (e.g.  hostile  interference,  electronic  countermeasures)  can  limit 
feedback  on  mission  performance  and  prevent  real-time  mission  intervention  during  remote  control 
operations.  This  may  necessitate  detailed  mission  planning,  including  contingencies  for  restricted 
autonomous  operations  when  human  supervision  and  authorisation  is  denied. 

Human  involvement  is  required  in  military  operations  to  direct  and  plan  the  use  of  military  capability,  and 
to  ensure  lawfully  correct  use  of  lethal  force.  This  is  achieved  through  the  application  of  human  command 
authority,  responsibility  and  accountability,  and  competency.  With  autonomous  UAVs,  some  of  that 
responsibility  is  delegated  to  increasingly  competent  computer  controlled  machines,  but  the  authority  and 
accountability  for  the  delegation  ultimately  remains  with  humans.  Ensuring  the  correct  human 
involvement  in  UAV  operations  provides  issues  for  Command  and  Control  (C2),  concept  of  operations 
(CONOPS),  ROE  and  for  the  specific  information  display  and  control  requirements  in  the  context  of  use 
i.e.  ISR,  combat,  or  support  roles.  For  military  relevance,  UAV  autonomy  concepts  must  be  integrated 
with  the  C2  requirements  of  both  national  and  international  C2  infrastructures  (joint  and  coalition 
operations).  C2  is  rooted  in  human  authority,  responsibility  and  accountability,  will,  leadership  and 
competency  in  judgement  and  decision  making.  The  potential  for  fully  autonomous  UAV  operations 
presents  significant  challenges  for  concepts  and  principles  of  military  C2.  UAV  control  requirements  need 
to  be  integrated  with  C2  frameworks  and  architectures  (information  flows,  decision  nodes,  and  dynamic 
interactions),  chains  of  command  and  CONOPS.  This  is  to  ensure  leadership  and  the  correct  delegation  of 
human  authority,  responsibility  and  accountability,  and  the  necessary  dynamic  human  interactions,  with 
appropriate  levels  of  trust. 

COGNITION,  CONATION,  MORAL  AND  ETHICAL  ISSUES. 

UAVs  are  used  extensively  to  gather  information  in  ISR  roles  for  human  interpretation.  ISR  information  is 
inherently  incomplete  and  uncertain.  Fundamentally,  computer-based  information  processing  systems  are 
limited  in  that  they  can  not  comprehend  the  meaning  of  information  in  both  human  cognitive  (e.g.  apply 
knowledge,  understand,  judge  consequences)  and  human  conative  terms  (feel  truth,  appreciate  human 
implications,  experience  emotions,  and  moral  and  ethical  values).  Critical  military  judgement  is  needed  to 
interpret  the  meaning  of  ISR  information.  Crucially,  UAVs  can  not  appreciate  the  effects  of  use  of  lethal 
force.  An  “emotional  connectivity”  is  needed  to  appreciate  the  “moral  value  of  killing  and  the  value  of 
human  life”  [2].  Critical  military  judgement  is  needed  for  decisions  on  use  of  lethal  force.  Failure  to 
ensure  proper  human  involvement  risks  rendering  UAVs  as  unusable  tools  for  military  purposes. 

The  study  of  moral  issues  (ethics)  is  “ concerned  with  or  relating  to  the  distinction  between  good  and  bad 
or  right  and  wrong  behaviour ”  (based  on  the  definition  of  ‘moral’  from  the  Oxford  English  Dictionary). 
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In  considering  moral  issues  of  using  autonomous  UCAVs,  Air  Chief  Marshall  Sir  Brian  Burridge  refers  to 
the  term  “ morality  of  altitude ”  that  was  coined  in  to  reference  the  disconnection  of  the  pilot  at  10,000  feet 
from  the  destruction  caused  by  bombing  on  the  ground  [2],  This  disconnection  led  to  a  lower  incidence  of 
psychological  problems  amongst  USAF  pilots  than  their  US  Army  colleagues  on  the  ground  during  the 
Vietnam  conflict.  He  believes  that  the  “ morality  of  altitude ”  is  at  the  heart  of  the  debate  of  how 
international  law  will  interpret  robotic  warfare  in  the  future.  He  concludes  that  “ Feeling  the  granularity  of 
the  battle-space  is  the  key  issue  in  interpreting  the  Rules  of  Engagement” . 

The  Air  Chief  Marshall  poses  the  future  possibility  of  the  “Play  Station”  operator  who  may  never  have  had 
actual  combat  experience,  no  connections  with  other  operational  units,  and  no  shared  operational 
experiences  [2].  Furthermore,  he  expects  that  future  highly  autonomous  systems  will  be  reliant  on  an 
experienced  programmer  for  their  autonomy,  who  may  not  have  any  experience  of  combat  operations  in  a 
manned  platform.  He  notes  that  this  will  ‘  further  remove  the  remote  pilot  from  the  system  and  place  him 
within  the  industrial  or  military  support  base”.  The  Air  Chief  Marshall  discusses  how  this  exacerbates 
“the  disconnection  of  air  power  from  the  shared  battle-space” .  In  considering  the  increasing  lethality  and 
persistence  of  UAVs,  he  questions  how  we  stop  the  “ Play  Station ”  generation  becoming  the  “ playground 
bully”  of  the  battlefield?  He  asks  if  this  disconnection  exacerbates  the  potential  for  the  “ play  ground  bully” 
in  all  of  us  to  emerge.  He  contrasts  the  simplicity  of  “ drop  and  drag”  mouse  actions  on  a  lap  top  during 
remote  reach-back  operations,  with  the  consequences  on  the  other  side  of  the  world.  One  of  the 
implications  of  these  issues  is  that  there  with  the  emergence  of  increasingly  autonomous  UMVs,  there  is 
an  increasing  need  to  prioritise  both  cognition  and  conation  as  human  dimension  drivers  of  HSI. 


4.  HUMAN  SYSTEM  INTEGRATION 

As  a  speciality,  HF  is  traditionally  concerned  with  the  study  of  the  man-machine  interface.  This  also 
includes  consideration  of  the  equipment,  the  physical  environment,  the  tasks  and  the  individuals  who  do 
the  work.  Humans  are  involved  throughout  the  UAV  life  cycle,  from  conceptualisation,  specification, 
design  and  development,  through  command,  control,  operation  and  maintenance,  to  decommissioning.  The 
term  Human  Systems  Integration  (HSI)  is  increasingly  used  in  NATO  nations  to  cover  the  broad  scope  of 
human  considerations  needed  from  a  human-centred  approach  to  systems  engineering  (or  in  a  system-of- 
systems  approach).  The  following  definition  of  HSI  has  been  agreed  by  NATO  NSA  Aircrew  Integration 
Panel  for  addition  to  AAP-45  (NATO  Glossary  of  Aircraft  -  Aircrew  Integration  (AI)  Specialist 
Terminology  and  Abbreviations):  The  technical  process  of  integrating  the  interdependent  elements  of 
Human  Factors  Engineering ,  Manpower,  Personnel,  Training,  System  Safety,  Health  Hazards, 
Survivability,  and  Habitability  into  the  system  acquisition  process  to  ensure  the  safe  and  effective 
operability  and  supportability  with  minimised  Life  Cycle  Cost  (LCC).  UAVs  change  the  challenges  of 
system  safety,  health  hazards,  survivability  and  habitability,  reducing  risks  compared  with  manned 
vehicles,  particularly  for  remote  “reach-back”  operations.  Otherwise  the  HSI  domains  of  HF  Engineering, 
Manpower,  Personnel,  Training,  remain  as  ever  highly  relevant  for  the  UAV  system  life  cycle. 
Notwithstanding,  achieving  the  correct  human-automation  integration  is  a  key  HSI  challenge  for  UAVs, 
with  significant  implications  for  HF  Engineering,  Manpower,  Personnel  and  Training. 


5.  MANNING 

Generally,  UAVs  are  expected  to  augment  the  force  and  to  create  potential  savings  in  human  resources, 
manning  levels  and  training.  However,  Air  Chief  Marshall  Sir  Brian  Burridge,  describes  how  the  current 
manpower  burden  of  remotely  piloted  operations  is  significant,  and  should  not  be  underestimated  [2].  The 
Air  Chief  Marshall  reports  that  as  part  of  the  Predator  Task  Force  at  Nellis  Air  Force  Base,  the  RAF  mans 
a  single  Predator  A  Orbit  in  support  of  the  coalition  intelligence,  surveillance  and  reconnaissance  effort. 
This  RAF  unit,  115  Flight,  totals  44  RAF  personnel.  He  explains  that  a  Predator  A  can  orbit  for  20  hours 
and  requires  2  crew  who  operate  for  8  hours  each,  totalling  6  crew  for  a  single  Predator.  In  addition,  the 
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operation  involves  analysts,  data  link  managers,  engineers  in  the  deployed  location,  and  the  crews  required 
to  launch  and  to  recover  the  UAV  in  theatre.  This  corresponds  to  a  considerable  manpower  intensive 
effort,  in  stark  contrast  with  the  current  aspiration  of  UAVs  to  reduce  the  manpower  burden.  In  the  future, 
UMVs  may  be  expected  to  operate  with  increased  levels  of  autonomy,  with  concomitant  reductions  in 
human  involvement  in  platform  control.  Estimates  of  savings  require  comparisons  with  manned  aircraft 
operations  providing  the  same  level  of  persistence.  But  on  the  evidence  from  Predator  A  operations,  we 
should  be  careful  not  to  underestimate  the  human  resources  requirements  of  UAV  operations. 

The  Global  Hawk  High  Altitude  Endurance  UAV  system,  with  relatively  higher  levels  of  autonomous 
functioning  compared  with  Predator,  has  ground  control  facilities  comprising  two  elements,  the  Launch 
and  Recovery  Element  (LRE)  and  the  Mission  Control  Element  (MCE).  The  LRE  accommodates  two 
persons  and  is  responsible  for  pre-flight  and  post-flight  ground  operations  and  the  takeoff  and  landing 
phases  of  flight.  The  MCE  accommodates  four  persons  and  is  responsible  for  the  mission  portion  of  the 
flight,  when  the  vehicle  is  at  cruise  altitude. 

Currently,  it  is  a  priority  in  many  NATO  Nations  UAV  research  programmes  to  reduce  the  manpower 
burden  by  reducing  the  ratio  of  operators  to  vehicles  for  flight  and  mission  control.  A  common  aim  is  to 
increase  operator  effectiveness  by  enabling  a  single  operator  to  control  multiple  UAVs  simultaneously 
(typically  up  to  four)  by  introducing  increased  levels  of  automation,  operator  decision  aiding  and  advanced 
human-computer  interfaces  (HCI). 

Air  Chief  Marshall  Sir  Brian  Burridge  considers  that  softer  human  issues  associated  with  operator 
selection  and  training  need  to  be  addressed  urgently,  ahead  of  some  of  the  technological  issues.  Predator  is 
remotely  operated  by  a  pilot  and  a  sensor  operator.  Other  UAVs  use  a  computer  operator.  The  Air  Chief 
Marshall  expresses  concern  that  without  proper  training,  operators  “could  be  faced  with  the  very  real 
possibility  of  unwrapping  one  of  these  systems  for  the  first  time  on  operations”.  Integration  and  interaction 
with  civilian  airspace  constraints  is  a  key  training  issue.  He  emphasises  the  importance  of  previous 
military  experience  gained  in  operations.  The  experience  of  operating  manned  platforms  enables  them  to 
interact  with  other  units  and  to  operate  safely  within  airspace.  He  notes  that  they  understand  the  needs  of 
other  units  through  this  shared  connection.  Solutions  may  need  to  be  found  in  the  selection  of  personnel 
with  appropriate  operational  experience,  and  in  creating  an  appropriate  work  context  for  proper  operator 
task  engagement  through  a  combination  of  HF  engineering,  HCI  and  training  in  RoE  and  effects 
appreciation. 

6.  INCREASED  COGNITIVE  COMPLEXITY 

In  the  future,  the  possibility  of  increasingly  autonomous  UAVs  can  be  expected  to  place  greater  cognitive 
demands  on  the  operator,  with  little  or  no  manual  control  required.  Basic  military  skills  and  knowledge 
will  continue  to  be  important,  such  as  airmanship  and  seamanship.  But  UAV  system  user  requirements  are 
driving  UCS  specifications  to  allow  one  operator  to  control  multiple  UAVs  (typically  4  UAVs  per 
operator  is  the  design  aim  for  advanced  systems).  Thus,  the  role  of  psychomotor  abilities  will  become 
diminished.  Performance  of  tasks  that  are  likely  to  be  required  include: 

•  Managing  and  controlling  multiple  UAV  missions 

•  Co-ordination  and  de-confliction  of  multiple  UAV  assets 

•  Interpreting  and  integrating  command  strategic  intent,  RoE  and  mission  control  requirements 

•  Recognizing  and  dealing  with  degraded  system  functionality 

•  Regaining  SA  after  loss  of  UAV  data  links 

•  Interpreting  displays  containing  multiple  UAV  perspectives 

•  Shift  of  system  control  to  other  team  members  or  control  stations 

•  Team- working  and  interpersonal  interaction 
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The  emphasis  will  be  on  the  UAV  operator  as  a  mission  manager,  on  multi-task  management  and 
performance,  on  judgement  and  decision-making  skills,  and  on  the  cognitive  ability  to  integrate  and 
interpret  dynamic,  complex  data,  in  order  to  make  rapid  and  effective  decisions. 


7.  REMOTENESS  AND  CONNECTIVITY 

From  a  system-of-systems  point  of  view,  the  term  “unmanned”  is  potentially  misleading.  It  is  most 
certainly  inappropriate  from  an  HF  engineering  perspective.  It  suggests  an  absence,  or  a  reduction,  in 
human  involvement,  and  consequently  a  lack  of,  or  a  lessening,  in  human  system  issues.  This  is 
particularly  unfortunate  since  the  opposite  is  probably  nearer  the  truth.  UAVs  remove  humans  from  the 
vehicle  (or  platform)  and  the  hazardous  operating  environment.  However,  UAVs  do  not  remove  humans 
from  the  system  of  use.  At  the  present  time,  with  human-in-the-loop  control,  advances  in  autonomous 
vehicle  technologies  are  worthless  without  an  effective  and  efficient  operator  remote  control/display 
interface. 

Generally,  separating  operators  from  the  context  of  use  risks  disconnection  from  the  battle-space,  reduced 
SA,  and  creates  difficulty  in  decision  making  and  in  maintaining  the  level  of  control  and  feedback  on  the 
effects  of  use.  Rather  than  reducing  the  human  system  issues,  increasing  remoteness  may  risk  reducing  the 
operator’s  capability  to  provide  effective  task  engagement,  situation  appreciation  and  timely  interventions. 
Increased  levels  of  autonomy  may  reduce  some  of  the  human-in-the-loop  workload,  but  autonomy  risks 
the  effects  of  disconnection  identified  above.  Research  is  needed  both  on  AI  techniques  for  autonomy  and 
on  HF  of  supervisory  control.  The  risk  of  disconnection  raises  the  importance  of  HF  engineering  for 
enabling  supervisory  control  and  for  exploiting  the  potential  mitigations  afforded  by  advanced  HCI 
design,  augmented  cognition  technologies,  SA  tools  and  operator  decision  support  aids.  HCI  style  guide 
information  is  available  for  interoperability  between  UCS  in  NATO  STANAG  4586.  HCI  is  a  rapidly 
advancing  field  and  research  is  needed  to  provide  properly  validated  advanced  HCI  solutions  for  future 
improved  UCS. 


8.  RESPONSIBILITY  AND  ACCOUNTABILITY 

It  has  been  suggested  that  UAVs  may  shift  the  balance  of  responsibility  and  accountability  for  UAV 
behaviours  and  effects  from  users’  decisions  during  systems  operation  towards  engineers’  decisions  during 
system  design.  The  development  of  this  argument  could  depend  on  technological  developments  affecting 
the  future  possibility  of  machines  that  never  make  mistakes,  the  levels  of  automation  employed,  and  the 
methods  of  supervisory  control  of  operations  and  effects.  International  humanitarian  law  on  military  use  of 
lethal  force  in  conflict  seems  likely  to  keep  responsibility  and  accountability  firmly  with  the 
user/commander  [2,  3].  So,  with  increasing  levels  of  autonomy,  the  need  for  system  transparency  and  SA 
could  grow.  User  involvement  in  systems  requirement  specification  will  become  increasingly  important  to 
ensure  that  critical  military  judgement  can  be  properly  exercised  in  the  context  of  use.  This  could 
necessitate  real  time  user/commander  control  of  the  level  of  automation  (i.e.  adjustable,  variable  levels  of 
automation),  in  addition  to  supervisory  control  of  the  specific  UAV  operations  and  effects. 


9.  ASSISTANT  TECHNOLOGY 

In  the  future,  rather  than  coping  with  unreliable  human  supervisory  control,  or  simply  removing  the 
operator  from  the  control  loop  entirely,  the  paradigm  for  operator  control  will  need  to  progress  to  one 
based  on  human-computer  co-operation,  as  implemented  in  advanced  pilot  assistance  systems.  Future 
UAVs  will  contain  associate  systems  (cf  virtual  cognition)  that  will  enable  the  UAV  operator  and  the 
associate  to  form  a  team  of  two  crewmembers  -  one  human  and  one  electronic.  This  partnership  is 
cognitive  co-operation  [5].  Ensuring  the  success  of  this  necessary  partnership  presents  significant 
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challenges  for  HF  of  UAV  systems.  Research  has  shown  how  real-time  F1F  engineering  of  variable  levels 
of  automation  or  adjustable  levels  of  autonomy  are  important  for  controlling  multiple  autonomous  UAVs, 
and  provide  the  key  to  developing  an  adaptive  human-computer  decision  partnership.  Consideration  is 
needed  of  the  relevance  of  mission  and  decision  enabling  technologies  for  augmenting  engagement.  These 
include  advanced  human-computer  interfaces,  multi-modal  virtual  media  “immersive”  synthetic 
environments,  task  and  user  monitoring  and  modelling,  collaborative  technologies  and  communication 
techniques  such  as  semantic  information/knowledge  web  approaches  to  decision  effectiveness. 
Visualisation  of  UAVs,  such  as  in  the  US  AFRL  “Vigilant  Spirit”  Research  UCS  support  for  air-to-air 
refuelling,  can  be  provided  to  augment  operator  task  engagement,  enhance  operator  situation  awareness 
and  trust  in  automation  through  direct  virtual  perception,  and  provide  intuitive  and  integrated  pictorial 
information  to  support  operator  supervision  and  intervention  decision  making. 

10.  CONCLUSION 

Virtual  media  are  extensively  exploited  by  UAV  systems.  Virtual  media  support  user  interaction  with 
UAV  systems  to  enable  UCS  supervisory  control.  Increasing  levels  of  autonomy  are  being  implemented  in 
UAV  systems.  High  levels  of  autonomy  present  challenges  for  maintaining  effective  supervisory  control 
with  virtual  media  UCS.  Ideally,  a  flexible  approach  is  needed  that  allows  a  variable  level  of  human 
intervention  and  autonomy,  with  the  need  for  “drill-down”  judged  in  real-time.  For  efficient  and  effective 
mission  supervision  and  discrimination,  the  operator/supervisor  needs  to  be  able  to  bring  added  value  to 
the  understanding  of  the  situation.  To  add  value,  he/she  will  need  to  be  able  to  use  knowledge  (e.g.  RoE, 
situation  awareness,  tactics)  and  to  take  into  account  additional  contextual  decision  information  not 
available  to  the  UAV  information  processing  system.  Otherwise,  the  level  of  supervision  may  be 
uncritical  and  lack  any  real  operator  decision  input,  with  the  resultant  legal  implications.  One  example  to 
avoid  would  be  authorising  target  prosecution  for  autonomous  UAVs  based  only  on  pre-set  automatic 
target  recognition  (ATR)  criteria  without  independent  operator  verification  of  the  target  context  RoE.  To 
mitigate  this,  the  C2  system,  UCS  and  virtual  media  need  to  provide  a  rich  operating  picture  for  mission 
assessment  and  appropriate  mission  performance  critiquing  tools. 

Force  augmentation  issues  relevant  to  the  human  operator  have  been  shown  to  exist  on  several  levels, 
including  individual  UAV  control  station  design,  vehicle  interoperability,  and  integration  of  UAVs  with 
manned  systems.  For  UAVs  to  be  successful,  they  must  be  fully  integrated  with  manned  systems  so  as  to 
enhance  the  strength  of  the  overall  force.  The  HFM-078/RTG-017  Technical  Team  have  identified  the 
following  priority  areas  requiring  further  research  to  determine  how  UMV  systems,  including  UAVs,  can 
be  developed  and  improved  further  as  force  multipliers. 

Interoperability 

•  Influence  of  the  environment; 

•  Accessing  and  navigating  large  data  sets; 

•  Common  intent; 

•  Data  latency  and  trust; 

•  Migrating  control  between  operators  and  teams; 

•  Operator  skills  &  embedded  training; 

•  Commonality  in  control/display  interfaces; 

•  Standardised  communication  protocol; 

•  Common  machine  understanding 

Framework  for  force  multiplication 

•  Control  theory; 

•  Hierarchy; 
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•  Sensing  world/own  state,  other  actors,  and  understanding  mission  objectives; 

•  Descriptive  analyses; 

•  Design  philosophy  &  guidelines 

Control  stations  -  Intelligent  operator  support 

•  User-centred  design; 

•  Mission  specific  multi-modal  interfaces; 

•  Intelligent  displays; 

•  Human-automation  shared  SA  &  intent; 

•  Knowledge  based  operator  assistant  systems 

Collaborative  work 

•  Coalition  history; 

•  Control  of  UAVs  -  locus  of  control  now  centrist,  future  distributed; 

•  Collaborative  work  environments  -  tools,  self  synchronisation,  teaming; 

•  Intelligent  cooperative  systems  -  collaborating,  automation  as  team  member; 

•  Nano  cognetics  -  bottom-up,  emergent  organising  principles  based  on  least  parsable  information 
exchanges 

Controlability 

•  Automation  is  not  the  same  as  autonomy,  since  the  latter  implies  that  control  is  relinquished. 

•  A  high  level  of  automation  does  not  mean  a  low  level  of  interaction,  but  it  opens  the  possibility  of 
interacting  and  controlling  at  higher  levels. 

•  In  order  to  maintain  military  relevance,  it  is  necessary  that  humans  remain  in  charge.  This  is  the 
axiom  of  human  control. 

•  Since  every  mission  must  consider  how  events  develop  over  different  time  frames,  control  always 
takes  place  on  several  levels  simultaneously. 

•  Humans  are  more  important  for  coordination  between  layers  than  for  action  within  each  layer. 

•  The  possible  contradiction  between  automation  assistance  and  human  ability  to  maintain  control 
must  be  prevented  by  a  human-centred  system  design. 
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